Skip to content

optimize data branch merge apply execution#24860

Merged
mergify[bot] merged 4 commits into
matrixorigin:mainfrom
gouhongshen:codex/data-branch-merge-exec-route-main
Jun 8, 2026
Merged

optimize data branch merge apply execution#24860
mergify[bot] merged 4 commits into
matrixorigin:mainfrom
gouhongshen:codex/data-branch-merge-exec-route-main

Conversation

@gouhongshen

Copy link
Copy Markdown
Contributor

What type of PR is this?

  • API-change
  • BUG
  • Improvement
  • Documentation
  • Feature
  • Test and CI
  • Code Refactoring

Which issue(s) this PR fixes:

Related issue #24099

What this PR does / why we need it:

This is the main cherry-pick of #24858.

DATA BRANCH MERGE materializes row differences into __mo_diff_* helper tables, then applies them to the base table with base-table DML that reads those helper tables.

runSql previously routed every statement mentioning __mo_diff_* through BackgroundExec. That was needed for temp-table DDL and temp-table-targeting DML, but it was too broad for the base-table delete/insert apply statements. On a 100M-row table with only 1000 updated rows, the apply phase spent seconds on the slower path.

This change keeps BackgroundExec for data-branch temp table DDL and DML that targets __mo_diff_* tables, while allowing base-table DML that only reads those helper tables to use the internal SQL executor.

Validation on main:

  • make cgo
  • CGO_CFLAGS="-I$(pwd)/thirdparties/install/include" go test ./pkg/frontend

@matrix-meow matrix-meow added the size/M Denotes a PR that changes [100,499] lines label Jun 5, 2026
@mergify mergify Bot added kind/bug Something isn't working kind/enhancement kind/test-ci labels Jun 5, 2026
@gouhongshen gouhongshen changed the title [codex] optimize data branch merge apply execution optimize data branch merge apply execution Jun 5, 2026
@gouhongshen gouhongshen marked this pull request as ready for review June 5, 2026 07:35
@gouhongshen gouhongshen requested a review from XuPeng-SH as a code owner June 5, 2026 07:35
@qodo-code-review

Copy link
Copy Markdown

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

@qodo-code-review

Copy link
Copy Markdown

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

@mergify mergify Bot added the queued label Jun 8, 2026
@mergify

mergify Bot commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Merge Queue Status

  • Entered queue2026-06-08 03:27 UTC · Rule: main
  • Checks skipped · PR is already up-to-date
  • Merged2026-06-08 03:28 UTC · at 6047b746365ce19148558bdb07d7a7bd83095aa2 · squash

This pull request spent 17 seconds in the queue, including 3 seconds running CI.

Required conditions to merge
  • #approved-reviews-by >= 1 [🛡 GitHub branch protection]
  • #review-threads-unresolved = 0 [🛡 GitHub branch protection]
  • github-review-decision = APPROVED [🛡 GitHub branch protection]
  • any of [🛡 GitHub branch protection]:
    • check-success = Matrixone Compose CI / multi cn e2e bvt test docker compose(PESSIMISTIC)
    • check-neutral = Matrixone Compose CI / multi cn e2e bvt test docker compose(PESSIMISTIC)
    • check-skipped = Matrixone Compose CI / multi cn e2e bvt test docker compose(PESSIMISTIC)
  • any of [🛡 GitHub branch protection]:
    • check-success = Matrixone Standlone CI / Multi-CN e2e BVT Test on Linux/x64(LAUNCH, PROXY)
    • check-neutral = Matrixone Standlone CI / Multi-CN e2e BVT Test on Linux/x64(LAUNCH, PROXY)
    • check-skipped = Matrixone Standlone CI / Multi-CN e2e BVT Test on Linux/x64(LAUNCH, PROXY)
  • any of [🛡 GitHub branch protection]:
    • check-success = Matrixone Standlone CI / e2e BVT Test on Linux/x64(LAUNCH, PESSIMISTIC)
    • check-neutral = Matrixone Standlone CI / e2e BVT Test on Linux/x64(LAUNCH, PESSIMISTIC)
    • check-skipped = Matrixone Standlone CI / e2e BVT Test on Linux/x64(LAUNCH, PESSIMISTIC)
  • any of [🛡 GitHub branch protection]:
    • check-success = Matrixone CI / SCA Test on Ubuntu/x86
    • check-neutral = Matrixone CI / SCA Test on Ubuntu/x86
    • check-skipped = Matrixone CI / SCA Test on Ubuntu/x86
  • any of [🛡 GitHub branch protection]:
    • check-success = Matrixone CI / UT Test on Ubuntu/x86
    • check-neutral = Matrixone CI / UT Test on Ubuntu/x86
    • check-skipped = Matrixone CI / UT Test on Ubuntu/x86
  • any of [🛡 GitHub branch protection]:
    • check-success = Matrixone Compose CI / multi cn e2e bvt test docker compose(Optimistic/PUSH)
    • check-neutral = Matrixone Compose CI / multi cn e2e bvt test docker compose(Optimistic/PUSH)
    • check-skipped = Matrixone Compose CI / multi cn e2e bvt test docker compose(Optimistic/PUSH)
  • any of [🛡 GitHub branch protection]:
    • check-success = Matrixone Standlone CI / e2e BVT Test on Linux/x64(LAUNCH,Optimistic)
    • check-neutral = Matrixone Standlone CI / e2e BVT Test on Linux/x64(LAUNCH,Optimistic)
    • check-skipped = Matrixone Standlone CI / e2e BVT Test on Linux/x64(LAUNCH,Optimistic)
  • any of [🛡 GitHub branch protection]:
    • check-success = Matrixone Upgrade CI / Compatibility Test With Target on Linux/x64(LAUNCH)
    • check-neutral = Matrixone Upgrade CI / Compatibility Test With Target on Linux/x64(LAUNCH)
    • check-skipped = Matrixone Upgrade CI / Compatibility Test With Target on Linux/x64(LAUNCH)
  • any of [🛡 GitHub branch protection]:
    • check-success = Matrixone Utils CI / Coverage
    • check-neutral = Matrixone Utils CI / Coverage
    • check-skipped = Matrixone Utils CI / Coverage

@mergify mergify Bot merged commit 8ad7a3f into matrixorigin:main Jun 8, 2026
23 of 24 checks passed
@mergify mergify Bot removed the queued label Jun 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/bug Something isn't working kind/enhancement kind/test-ci size/M Denotes a PR that changes [100,499] lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants